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Abstract 

We consider a simple optimal probabilistic problem solving strategy that searches through 
potential solution candidates in a specific order. We are interested in what impact has in¬ 
terchanging the order of two solution candidates with respect to this optimal strategy on the 
problem solving time. Such interchange (i.e., error) can happen in the applications with only 
partial information available. We derive bounds on the excessive problem solving time due to 
these errors in general case as well as in some special cases in which we impose restrictions on 
the solution candidates. 
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1 Introduction and layout of the paper 


Problem solving is probably the most common mental activity of humans, and if artificial in¬ 
telligence is to succeed in creating human like agents, it must deal with the design of general 
problem solving algorithm. One reason why the human like agents are desirable in the first 
place is the effectivity with which the human solvers manage to solve vast amount of problems. 
Moreover, the same human brain (i.e., the same underlying mechanisms) can solve problems 
from a very wide variety (mathematics, logic, pattern recognition, orientation in space, coordi¬ 
nation to name just a few). Solomonoff [13] in his work on general system for solving problems 
described a basic model of problem solving process. 

There is a casino with a set of bets each winning the same prize. The k*^ bet has probability 
of winning pk and costs dk dollars. All probabilities are independent, and one cannot make the 
same bet twice. The probabilities pk do not need to sum to 1 (i.e., there can be three bets 
with 50% chance of winning). If all bets cost equally (e.g. one dollar), then, obviously, the 
best strategy is to take the bet with highest win probability available. If not all are same, 
then by selecting the bet with highest ratio Pk/dk available the expected money spent before 
winning will be minimal (proof given below). By changing dollars dk to time tk we get the least 
expected time before winning. 

The problem solving interpretation of this scenario is straightforward, and it outlines the 
optimal probabilistic problem solving strategy provided we know the values pk and tk for all so¬ 
lution candidates (i.e., probability of solving the problem and time of execution for each solution 
candidate). Admittedly, this requirement of knowing pk and tk for all solution candidates is very 
strong and generally hard to meet. However, in our paper Theory of the effectivity of human 
problem solving submitted for publication we argued that humans have access to these values or 
at least to their approximations and that they use them (consciously or unconsciously) to drive 
their problem solving according to the Solomonoff optimal strategy. It is for this reason we are 
interested in the Solomonoff optimal problem solving strategy. Moreover, since humans do not 
always follow the Solomonoff optimal problem solving strategy (inaccurate approximations of 
Pk or tk, or other reasons), we are interested in what effect the interchange of two candidates 
has on the problem solving time. 

The importance of automatic selection of the problem solving methods the has long been 
noted laiHiE]. For example, Fink [T] developed a statistical method for automatic selection 
among methods based on analysis of their past performance. More particularly, Fink analyzed 
the expected gain (there is a reward for solving the problem) of each available method based 
on its past success’, failures and the corresponding execution times. The proposed system for 
solving problems not only chooses the most efficient method (given its limited available data 
and a time bound), but also decides when to abandon the chosen method if it takes too much 
time. 

Solomonoff’ in his work on the general system for solving problems used his theory of 
inductive inference uni E] to approximate the probabilities pk and Levin search mm to 
approximately follow the optimal order of examination of candidates as outlined in his betting 
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scenario m- Since this is an approximation of the optimal order, it does not need to follow it 
exactly, and we might be interested in what effect the shuffling of candidates has on the problem 
solving time. Other problem solving systems that are also related to the Solomonoff problem 
solving strategy and the approximation of probabilities pk (e.g., AIXI proposed by Hutter Ell!) 
may benefit from this paper in the similar way. 

Schmidhuber [9| described a similar notion of time optimality of the problem solving. While 
he also uses the values pk and tk, he does not try to approximate the probabilities pk (termed 
initial bias) as Solomonoff. Instead, he assumes that they are given as input (although they 
may change during the problem solving process; the same applies to the Solomonoff problem 
solving system). Again, the changes in the initial probabilities may lead to different order of 
candidate examination, and it is interesting to ask what effect does this have on the problem 
solving time. 

The layout of the paper is as follows. In Section\^we prove the optimality of the strategy 
mentioned in the introduction as Solomonoff m did not do so. The proof will serve as a basis 
for all our subsequent results. In S'ectionOwe consider the effect of interchanging two candidates 
with respect to the optimal strategy on the problem solving time. In Section\^we give several 
bounds on the error resulting from the mentioned interchange. However, since the values pk and 
tk can be arbitrary, we examine special cases under which reasonable bounds can be achieved. 
In Section [5| we consider a modification of this strategy when each solution candidate can have 
multiple values of execution time (i.e., multiple values tk). This modification models the case 
when we applied the same solution candidate (e.g., a method) to two or more similar problems 
each time solving the problem in different time. 

2 Theorem in probability 

Let there be a set of bets each winning the same prize. Let pk denote a probability of winning 
with bet, and let dk denote the cost of this bet. Each bet can only be taken once, and the 
values Pk do not need to sum to 1. One strategy to win is to take bets in the order given by 
decreasing value This strategy appeared in Solomonoff paper on general problem solving 
system m, but the proof was omitted. In this section we give proof of this statement because 
it will serve as a basis for all of our subsequent results. Additionally, since we would like to 
apply this strategy in real world, we restrict the number of bets to some finite number N. 

Theorem 2.1 (Solomonoff [l3]). Let there be a set of N bets each with probability 

of winning pk and cost dk- If one continues to select subsequent bets on the basis of maximum 
Pk/dk, the expected money spent before winning will be minimal. Suppose we change dollars to 
some measure of time tk. Then, betting according to this strategy yields the minimum expected 
time to win. 

Remark 2.2. Note that if the bets are selected in the order: 1^^, 2"’'^,..., then the prob¬ 
ability of using and winning with a particular bet k is not pk but Oti “ Pi) ' Pk- This is 
because in order to make and win with the bet all bets with the indices 1, 2,..., k — 1 must 
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have failed. 


Proof of the Theorem \2. h Without loss of generality, we may assume that the sequence of bets 
is ordered according to the values in the decreasing order, i.e.. 


ti t2 tk tN 


( 2 . 1 ) 


in which case the Solomonoff strategy SOL is {sk)k=i- Let Et be the expected time spent 
before winning using the Solomonoff strategy SOL. Clearly, 

N k k-1 

= X] ■ n 

k=l1=1 j=l 


We want to show that this strategy is optimal (with respect to the time spent before winning). 
Let ABC = ( 54)^1 be any betting strategy (i.e., a sequence of bets). Furthermore, let Eabc 
be the expected time spent before winning for the strategy ABC. Clearly, 

N k k-1 

Eabc = EE *-.'!! iX-Pij) -Pik- 

k=l 1=1 j=l 


Our aim is to show that Eabc ^ Et- !f ABC = SOL, then we have nothing to prove. Now 
assume that there are two immediately subsequent bets S 4 and in the sequence ABC such 
that ^ . The case when ^ for each a G {1, 2,..., — 1} but ABC ^ SOL will 

Ba Ua+1 Ba **4+1 

be considered below. Let ABC' be a modified sequence ABC in which the terms S 4 and 
are interchanged. We will show that Eabc > Eabc where Eabc denotes the analogous value 
for ABC'. First of all, notice that all terms in Eabc and Eabc are equal except for the terms 
on the af^ and (a + 1 )*^ position. In the expression Eabc we have the following value related 
to these two positions 


a a—1 a+1 a 


1=1 j=i 


1=1 j=i 


Pij ) ■ Pia + l > 


while in the expression Eabc there is 


^ a—1 


a—1 


'' a—1 


a—1 


+ l ■ Yli^-Pij) -Pia+l + +K+i+ti^ • - Pia + l)Pi. 


\l = l 

Therefore, 


f=i 


a=i 


j=i 


'' a—1 


a—1 


Eabc - Eabc = ( X ] | ' JI “ PL)' Pl- 

\i=i J j=i 

/a-l \ a-1 

- +b,+i • n (^-Pl) -Pia+iE 


\l=l 

^ a—1 




a—1 


ia+l 


+ + b, + ti ^+1 ■ “ PL^(^ - Pia)Pia 

\l=l J j=l 

( a-l \ a-l 

+^4+1 • Yli^ - Pij)^^ - Pia + l)Pia- 

1=1 J j=l 
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By substituting 


E 


and some rearranging we get 


a—1 


a—1 


Yjtii and n = n (^ 


■Pii 


1=1 


i=i 


Eabc 


Eabc = En • {Pia - Pi a+1 ) 


+ X\-{tiaPia -ti, + lPia+l) + 
+ -Pia) + 


+ n ■ {Pia + 1 - Pia) 

~ ■ {^iaPia + 1 ~ ^ia+lPia) ~ 


a—1 

= Yli'^ - Pij) ■ tiaK+l 
i=i 




> 0 . 


It follows that ABC can be turned into a betting strategy DEF = {smk)k=i ''^hh the property 
Pirn > > ... > Ehm. by repeatedly modifying the obtained sequences. Moreover, the 

^m2 

expected amount of dollars paid over a sequence (i.e., strategy) before winning is not increased 
after its modification. Thus, Eabc > Eabc > ••• > Eqef- If DEF ^ SOL, we proceed as 
follows. Let DEE' be a modified sequence DEF in which any two terms Sm^, and Sm(,+i with 
PpE = P”^b+i interchanged (call this kind of modification by simple modification). Using 
the same calculation as for Eabc — Eabc above, we get Eeef — E^ef' = 0 ; where E^ef' 
denotes analogous value for DEE', since 7 ^ — = 0. Now one can observe that DEF 

tmb ^"16+1 

can be turned into SOL by repeatedly modifying the obtained sequences (using only the simple 
modification). Thus, Eeef = E^ef’ = ••• = Esol- 

Finally, let us consider the case when ^ for each a € {1,2,..., N — 1}, but ABC 7 ^ 

SOL. In such case, we can turn ABC into SOL by the same way as we have turned DEF into 
SOL above, and therefore Eabc = Et- Consequently, Eabc > E^ef = Et (see above), or 
Eabc = Et- Hence, SOL is optimal, since the strategy ABC has been chosen arbitrarily. □ 


Remark 2.3. Notice that the expected betting time is not given by Et because we did not 
include the possibility that all of our bets failed. The corrected value Et is given by 

N k k-1 N N 

Et = -pk + ^ti-Ylil- pj). (2.2) 

k=l 1=1 j=l 1=1 j=l 

This is because the probability of each bet failing is O^i (f “ Pj) while it takes us altogether 
X)iLi ti amount time to discover this. 


3 General effects of exchanging the order of two candidates on 
the problem solving effectivity 

In this section we abandon the Solomonoff’ original betting scenario, and we will talk about 
solution candidates (i.e., bets) and problem solving strategy (i.e., betting on candidates). We 
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consider and qnantify the expected decrease in problem solving efFectivity when the solver makes 
an error and interchanges the order of two (not necessarily immediately subsequent) candidates 
with respect to the optimal Solomonoff problem solving strategy. 

Lemma 3.1 (Klamkin and Newman [5]). If xi,X 2 , ■■■,Xn are numbers in [0,1] whose sum is 
denoted by S, then 

n 

]J(1 - Xi) < e~^ . 
i=l 

Lemma 3.2 (Wu |14]i. Let 0 < Xj < 1, i = 1,2,n, n > 2, n € N. Then we have 

n 

n n / \ 

]^(l - a:*) > 1 - ^ Xi + (n - 1 ) I Xj I 

i=l i=l \i=l / 


Remark 3.3. For the rest of this paper we will use the following notation 


Sn. 

Prr 


V"" n- T 

UZlPi: Qr 




Theorem 3.4. Let ^ > 0 for some k (assuming to be ordered as before in 

Theorew A2.1\) . Then, following the optimal Solomonoff strategy from Theorem, \2.1\ with (/c + l)*^ 
solution candidate tried just before kf^ (a solver’s error) yields a sub-optimal expected amount 
of time spent before either finding a solution or discovering that none of our solution candidates 
works, and the expected excess EXC can he quantified as follows 

EXC = 11(1 - pj) • 44+1. 

Furthermore, 

44+1 • > EXC >(^- 44+1 -(l-Sk-i + ik- 2)P^A . 

V ^k-\-l J \^k ^/c+1 / \ / 


Proof. The theorem follows directly from proof of the Theorem \2. l\ (see the expression Eabc ~ Pabc 
and below). To derive the bounds, use Lemma, \3.1\ and Lemma E3 For notation see Remark 

roi □ 

Theorem 3.5. If {^)^i are ordered as before in Theorem \2.1[ then exchanging the k^^ and 
{k + n)^^ solution candidates in the optimal Solomonoff strategy from Theorem \2. 1\ (a solver’s 
error) increases the expected amount of problem solving time by at most the excess 

EXC = qi + q2 + 
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where 


Q.1 — 1 ‘ Qk—l ' {pk-\-n Pk) Qk—1 ‘ i^k-\-nPk-\-n ^kPk)^ 

k+n—1 


Q2= 'Pi{r 

l=k+l 


Pk Pk+n . \ 1 Pk-\-n 

\ \pk-\-n ^k) 


QS — * Qk^n—1 


'^-Pk 

Pk Pk-\-n 
'^-Pk 


^-Pk 


Proof. Use the equation (|2.2|) and subtract it from an analogous equation in which the and 
(fc + candidates are exchanged. The theorem follows by straightforward computation. □ 

Remark 3.6. The practicality of the Theorem \3.4\ \s considerably limited by the fact that the 
values and (U)^i can be (almost) arbitrary numbers (obviously, we want pi € [0,1] 

and ti > 0). Thus, we could not further simplify the terms qi,q 2 ,Q 3 - In Section^we make 
some assumptions about these numbers to obtain reasonable bounds on the term ECX from 
Theorem \3.4\ Note that these assumptions directly affect the quality of the obtained bounds 
and the form of the resulting formulas. On the other hand, they also reduce the generality of 
the derived bounds. 

4 More precise quantification of the effects in some special cases 

Lemma 4.1. Let r he any real number other than 1. Then it holds 

n 

= n -rl 2 • 

1=1 ^ > 


A very general bound of the EXC term from the Theorem \S. ,5l follows. Note that we were not 
able to derive similarly simple lower bound with the same weak assumptions (see Theorem 4-3). 

Theorem 4.2. For each i = 1,2 ,..., N, let c < pi < d for some c,d& (0,1) and 0 < ti < T for 
some T. Then, the term EXC from the Theorem \3.5\ can be upper bounded as follows 


EXC < T 


(1 _ c)^ (A - B(1 - c)"-M , 

1-pk c 


where 


, , 1 , 1 — Pfc c/c 

A = l + - + k + - - - -, 

^ 1 Pk-\-n 1 C 


b = (i-£) (t + „) + i. 
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Proof. Let qi, q 2 , and qs be expressions from Theorem \3. 5\ From (1 — pi) < (1 — c) for all i we 
have for qi 

0.1 — Qk—lPk+n {Tk—l P tk+n) Qk—lPk {Tk—1 tfc) ^ 


— Qk—iPk+n i'^k—l “b tk+n) — 

< (1 - cf-^dkT. 


Finally, we transform qi into 


^ rr^-Pk+nd . l-Pk ck 

qi < T— ---(1 - c) 


1 -Pfc c 


1 Pk+n 1 C 


For q 2 we have 


k+n-l . ^ 

92 < V Qi-mUr , n+’' + P w+n 'j 


= T 


1 Pfc+n 
^-Pk 

T 1 P/c+n 


/c+n —1 


Qi-i{t +i)m < 


/=/c+l 
fc+n—1 


<T—y ( 1 -c)^-^(/ + 1 ) 

- i-pfc ^' 


= T 


1 Pk+n 
l-Pk 


l=k-\-l 


t' n—1 


n—1 


<'(1 - E(i - h' + (*:+ 1 ) E(i - 


1=1 


1=1 


By using LemmojT^we can further write 

n—1 


y (1 - cYl = (1 + (c - nc - 1)(1 - c)-i) , 


^ = 1 

n—1 


(t+l)E(l-9)' = (fc+l)(l-c) 


1 - (1 - c)^-^ 


1=1 


Finally, we have for q 2 

< j. ^-Pk+n d _^^kf^^]2 + k + {l- c)"-i ( - A: - 1 

1 -pfcC VC VC 

^y l-Pfc+n rf(i_c)fc fl +- + A:-(1-cf-^ fA; + n+- 

I-Pk c VC V c 


(4.1) 


(4.2) 

(4.3) 


(4.4) 


For 53 we have 


which we transform into 


53 < (A; + n)r(l - c)'‘(l - c) 


, 1 Pk+n d , .f. 


kf-\ „\n-l 1 Pk+r. 


^-Pk 


qs < T \ ™ L (1 _ cf • (1 - cT-^-ik + n). 

d 


1 -Pfc c 

By putting (14.1|) . (14.41) . and (14.51) together the theorem follows. 


(4.5) 


□ 
















Theorem 4.3. For each i = 1,2, let c < pi < d for some c,d& (0,1) and 0 < t < ti < T 

for some t,T. Furthermore, let > Pk+n o,nd tk < tk+n, where Pk,Pk+mtk,tk+n o,re from 
Theorem AS.,5\ . Then, the term EXC from the Theorem Ad.,5\ can be lower bounded as follows 


where 


EXC > t——(1 _ df (A - B(1 - d)^-^) , 

1-pk d 


d Pk Pk+n 


dk 


l-d 


dt e ‘^'=-1 \ 
^(1 -d)^ ) ’ 


B = 



(k + n) + 


l-d 

d 


Proof. Let qi, q 2 , and q^ be expressions from Theorem. I,?.,51 We rewrite qi as 
Ql Qk—l ifhk—\ A~ tk.\-ri) Pk+n Qk—1 if^k—l A~ tk) Pk- 
By Lemma \3.1\ we have ^(1 — Pi) < and with ]^(1 — pi) > 0(1 ~ d) we can write 

> (1 - d)^^^ktc - e-^’^-^kTd 


which we transform into 

^ Pk - Pk+n c _ ^k f '^-Pk dk _ 1-pk dfkt e-^>‘-^ 

1 - Pfc d^ \pk - Pk+n l-d Pk- Pk+n cT (1 - d)^ ' 

The rest of the proof is analogous to the proof of the Theorem \4-^ 


□ 


A different bound of the EXC term from the Theorem Id. ,51 follows. In this case we chose a 
strong assumption that all L (see Theorem \2. R are equal. This case models a situation where the 
solver has candidates that take approximately the same amount of time (e.g., a weak/novice 
solver that only has very general methods to apply in problem solving that equally take a 
lot of time, or an expert solver that has a set a specific methods with comparable execution 
time). Note that in this case we state both upper and lower bound with the same assumptions 
(Theorem \4-4\ and Theorem \4.5\ respectively). 


Theorem 4.4. For each i = 1,2,..., N, let c < pi < d for some c,d(z (0,1) and ti 
some T > 0. Then, the term EXC in Theorem \3.5\ can be upper bounded as 

EXC < U - B{1 - c^-^) 

1-pkC 

where 


A = 1 + kc 


^-Pk C 

1 — c d 



T for 


B = 1 — c + c(n + /c) (1-- 

d 
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Proof. First of all, observe that with ti = T for all i = 1,2, the order of candidates 
from Theorem \2.1l which is given by depends only on pi. Thus, by m we have 

Pi >P 2 > ••• >PN, and particularly pk > Pk+n- 

Let qi, q 2 , and q^ be expressions from Theorem, \3.5\ With pk+n —Pk<0 and U = T for each 
i = 1, 2 ,N we have for qi 

qi = kT{pk+n -Pk)Qk-i, 
and because —Qm < — (1 — d)™, we can write 

qi < -kT{pk - Pfc+n)(l - 


which we transform into 


^ rj^rPk - Pk+n (1 - c)^ jl-Pk fl-d 

qi < -Td— - 2 - 11 - 

^ — Pk d 1 — c\l — c 


fc -1 


(4.6) 


For q 2 we have 


q2 = T 


Pk Pk-\-n 
I-Pk 

Pk Pk-\-n 


k-\-n—l 


^ Qi-ilpi < 


l=k-\-l 

k-\-n—l 


<T ^k-Pk+n ^ y 

1-pk ^ ^ ^ 


l=k-\-l 


/n—1 


n—1 


^ j,^ Pk yk+n ^,^ _ y -1 ^ /(I _ c)' + k ^(1 - c)M . 

\l=l 1=1 / 

By Lemma \4.1\ we can use similar equations to (14.21) and (|4.3p , and we can further write 


^2 < j,^Pk_^i)_^ . (1 + fcc - (1 - c)"-i (1 + c(n + A: - 1))) . 
l-Pk 

For g 3 we have 

q^ = {k + n)T ^\~ Qk+n-i < 


(4.7) 


< (A; + n)T 


1 -Pfc 


Pk Pk-\-r, 


which we transform into 

qs < Td 


1 -Pfc 
Pk - Pk+n (1 - cf 


(1-c) 


k+n — 1 


^-Pk 


d 


(A: + n)(l -c)"-L 


Finally, by putting (|4.6p . (14.711 . and (14.80 together the theorem follows. 


(4.8) 

□ 


Theorem 4.5. For each i = 1,2,...,A^, let c < pi < d for some c,d a (0,1) and ti = T for 
some T > 0. Then, the term EXC from the Theorem \3.5\ can be lower bounded as 

EXC > rg Pfc (1 - df ^ 

l-Pk d^ '■ 
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where 


A = 1 + kd 


I — Pkd e ‘^''-1 
l-d c (1 - d)^~^ 


B = 1 — d + d{n + k) 



Proof. By the same reason as in the beginning of the proof of the Theorem \4-4\ ''^6 have 
Pk Ppk+n- Let qi, q 2 , and q^ be expressions from Theorem, I,?. ,51 With pk+n — Pfc < 0 and 
ti = T for each i = 1, 2, N we have for qi 

qi = kT{pk+n - Pk)Qk-i, 

and by Lemm,a \S.l\ we can write 

qi > -kT{pk - pk+n)e~^'^-^ 

which we transform into 

„ Pk- Pk+n (l-d)*' , 1 - Pk 

-5^ ■ "TVd 

The rest of the proof is analogous to the proof of the Theorem \4.4\ n 

Finally, we examine a case in which the probabilities pi (see Theorem \2.1^ are all equal. 
This case models, for example, a situation where the solver has only general problem solving 
methods, and he is unable to distinguish which one is the most suitable. 

Theorem 4.6. For each z = 1,2, let Pi = p for some p E (0,1). Then, the term EXC 

from the Theorem \S.,5\ equals to 

EXC = (4+n - 4)(1 - pf-^ (1 + (1 - p) - (1 - pT) ■ 


Proof. Let gi, ^ 2 ? and qs be expressions from Theorem \S.5i Then, 

Ql — Qk—l{tk-\-'fi t/c) — (fk-\-n ^/c)(l p) •) 

fc+n—1 

Q2 ^ ^ Ql—lP{^k-\-n 

/=/c+l 

n—1 


p(4+n - 4)(1 -P)*" 


= p(tk+n - tk)il -p) 


1=1 

kl-{l-p) 


n—1 


p 


q3 = 0 . 

By summing the expressions for qi,q 2 , qs we get the final result. 


□ 
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5 The case with multiple values of tk 


In real life problem solving a particular solution candidate (e.g., a method) could have been used 
to solve multiple similar problems each time consuming a different amount of time. Therefore, 
when the solver is considering a potential solution candidate, it has one cumulative probability 
of success (e.g., based on the past experience and the strength of similarity/relatedness with 
the current problem model), but it can have multiple application times because of this possible 
application to the similar problems in the past. 

Theorem 5.1. Let Sk be a solution candidate which we in the past applied times, and let 
tk,j be the execution time of the application. Denote the mean execution time of the solution 
candidate s^ with Et^: 

_ ^fc,i + + tk,n^ 

nk 

If one continues to select subsequent candidates on the basis of maximum pk/Etk, then the 
expected time before solving the problem will be minimal (provided the problem can be solved by 
one of our candidates). 

Proof. By using the linearity of the expected value, the proof is identical to the proof of the 
Theorem \2.1[ □ 
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